Skip to content

fix(libc): follow measured wide printf behavior - #5303

Open
MarcZX10 wants to merge 6 commits into
boykopovar:mainfrom
MarcZX10:codex/wide-printf-extensions-5293
Open

MarcZX10 wants to merge 6 commits into
boykopovar:mainfrom
MarcZX10:codex/wide-printf-extensions-5293

Conversation

@MarcZX10

@MarcZX10 MarcZX10 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Updates non-secure guest wide printf formatting to match the measured literal-text behavior for %q, %D/%O/%U, unknown conversions, and width handling observed from libSceLibcInternal.sprx on firmware 12.70. The probe called swprintf from a payload process; it did not exercise a title or its bundled libc.prx, whose Dinkumware _Printf behavior remains unverified. The probe also showed that a trailing % read past the format terminator, so incomplete conversions continue to fail safely. The secure snwprintf_s path continues to reject these forms because its console behavior was not measured.

The original implementation commits retain Hussein Kadhim as author, and the original test commits retain MarcZX100. The review follow-up is authored by MarcZX10.

Tested

Linux x86-64, GCC 12.2.0, CMake 3.25.1, Ninja, Release:

  • Built guest_wide_formatting_tests; ctest -R guest_wide_formatting --output-on-failure: 1/1 passed.
  • python3 tools/check_conventions.py --base upstream/main --head HEAD: passed.
  • git diff --check upstream/main...HEAD: passed.

No title libc.prx runtime test, Windows build, or full test suite was run.

Checklist

  • Based on current main; no other open PR implements the same wide-format functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; silent stubs are listed under technical debt
  • No notes, investigation .md files or images added to the repository
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Which title prints %q, %D/%O/%U, an unknown conversion or a trailing % through the wide printf family? The source here is FreeBSD 11.4, but the console's libc.prx formats through Dinkumware's _Printf, which can handle unknown conversions and q differently (same question as the review on #3987). Until a title reaches it, or _Printf in a title's libc.prx confirms these cases, main's throw is the safer behaviour.

@MarcZX10

Copy link
Copy Markdown
Contributor Author

I do not have title traces or Dinkumware _Printf evidence for these wide-format cases. The current evidence is FreeBSD 11.4 behavior and Linux-side tests, so I cannot claim console compatibility for %q, %D/%O/%U, unknown conversions, or a trailing %. I agree this leaves the added behavior unverified against the console; absent title or _Printf evidence, main’s throw is safer. I will avoid presenting these cases as title-required until we have that evidence.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. Blocking points:

  • The console's libc.prx formats through Dinkumware's _Printf, not FreeBSD's vfwprintf. As discussed above, neither a title nor _Printf evidence shows that %q, %D/%O/%U, unknown conversions (with width) or a trailing % behave as FreeBSD 11.4 does there; main's throw is the safe behaviour until then. Evidence that would unblock it: the relevant branches of _Printf disassembled from a title's libc.prx (same approach as in #3987), or a title that reaches these cases.
  • Once evidence exists, the branch conflicts with main in core/libs/tests/GuestWideFormatting.cpp; it needs a rebase.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5021 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestWideFormatting.cpp
PR 6226 docs/dev/debt/libc.md
PR 6469 docs/dev/debt/libc.md
PR 6565 docs/dev/debt/libc.md
PR 6713 docs/dev/debt/libc.md
PR 6819 docs/dev/debt/libc.md
PR 7081 docs/dev/debt/libc.md
PR 7332 docs/dev/debt/libc.md
PR 7386 docs/dev/debt/libc.md
PR 7411 docs/dev/debt/libc.md
PR 7417 docs/dev/debt/libc.md
PR 7447 docs/dev/debt/libc.md
PR 7473 docs/dev/debt/libc.md
PR 7525 docs/dev/debt/libc.md
PR 7544 docs/dev/debt/libc.md
PR 7545 docs/dev/debt/libc.md
PR 7634 docs/dev/debt/libc.md
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 7807 core/libs/prx/libc/src/FormattingWide.cpp
PR 7897 docs/dev/debt/libc.md
PR 7952 docs/dev/debt/libc.md
PR 8023 docs/dev/debt/libc.md
PR 8037 docs/dev/debt/libc.md
PR 8087 docs/dev/debt/libc.md
PR 8129 docs/dev/debt/libc.md
PR 8137 docs/dev/debt/libc.md
PR 8148 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8226 docs/dev/debt/libc.md

@ionut27001

Copy link
Copy Markdown
Contributor

I have some hardware data for these cases, though not the kind the review asks for. @adriadam10: this is libSceLibcInternal.sprx's swprintf in a payload process on a PS5 (firmware 12.70), not _Printf in a title's libc.prx. I have no title trace and did not disassemble libc.prx, so this does not settle the blocking point.

Setup: an ELF payload (ps5-payload-dev/sdk v0.44, clang 18.1.8), started from a payload manager, resolves swprintf from libSceLibcInternal.sprx and calls swprintf(buf, 40, fmt, a1, a2, a3, 0, 0) with a 16-bit wchar_t. Each ASCII format is copied unit by unit, NUL included, into the same wchar_t fmt[48], which is never cleared. text is the output up to its first NUL (nul_at), logged as ASCII (? for any other unit; none appeared).

p23: meta fw=0x12700001 pid=135 sdk=v0.44 compiler=Clang 18.1.8 built=Oct 18 2026 02:46:33
p23: resolve sym=swprintf module=libSceLibcInternal.sprx exported=yes
p23: case meta sizeof_wchar_t=2
p23: case swprintf fmt="%qd|%qu|%qx|%5qd" ret=18 text="%qd|%qu|%qx|   %qd" nul_at=18
p23: case swprintf fmt="%D|%O|%U" ret=8 text="%D|%O|%U" nul_at=8
p23: case swprintf fmt="%hD|%hhU|%5D|%-4O|" ret=17 text="%D|%U|   %D|%O  |" nul_at=17
p23: case swprintf fmt="%y|[%5%]|[%-5y]" ret=18 text="%y|[    %]|[%y   ]" nul_at=18
p23: case swprintf fmt="%05y" ret=5 text="   %y" nul_at=5
p23: case swprintf fmt="100%" ret=15 text="100%5%]|[%y   ]" nul_at=15
p23: case swprintf fmt="%d%" ret=13 text="3%5%]|[%y   ]" nul_at=13
p23: case swprintf fmt="%lqd" ret=3 text="%qd" nul_at=3
p23: case swprintf fmt="%m" ret=2 text="%m" nul_at=2

Compared with CheckFreebsdConversions in core/libs/tests/GuestWideFormatting.cpp and the new line in docs/dev/debt/libc.md at 178de73:

  • q, %D, %O, %U print as written. h/hh are dropped, and the width pads the printed text: %5qd gives %qd, %5D gives %D and %-4O gives %O . The test expects -5|7|ff| 9 (13), 5000000000|10|18446744073709551615 (34) and 70000|70000| -3|10 | (23). My arguments differ from the test's (5, 300, 7 for the third format), but no argument appears in these three outputs.
  • Unknown conversion: %y keeps its %, and the width counts both characters: [%-5y] gives [%y ]. The test expects y|[ %]|[y ]|€ (19); I ran the format without |%€. [%5%] matches. %05y is blank-padded, as the debt line says, but it prints %y, not y.
  • Trailing %: the test expects 100 (3) for "100%" and 7 (1) for Format(buffer, 48, u"%d%", 7). Here swprintf printed the % and kept reading past the terminator. After 100%, fmt still held 5%]|[%-5y] from the earlier %y|[%5%]|[%-5y], and that tail appears formatted in the output. The same tail follows 3% in the %d% case, which also passed the terminator 100% had left. I ran this through swprintf, not vswprintf. What gets printed depends on the memory after the format, but it did not stop at the %.
  • %m and %lqd: the debt line says these "still fail", and the test expects a negative return (and an empty buffer for %m). Here they print as text: %m (2) and %qd (3).

Main does not match this output either: vswprintf_nid_postfix returns -1 with an empty buffer when FormatWide throws (FormattingWide.cpp on main), and every case above throws there. In this process, all of them came back as text.

Not measured: a title or its libc.prx; vswprintf and wprintf; all the snwprintf_s cases; the surrogate-pair cases; |%€, ab%5, %d%l, a lone % and %lp. Each case ran once.

I can attach the probe source and the full log if that helps.

@MarcZX10
MarcZX10 force-pushed the codex/wide-printf-extensions-5293 branch from 178de73 to db5c096 Compare October 19, 2026 15:00
@MarcZX10

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (00d7e611ef). Preserved all three commits and original authorship. git diff --check passes. I attempted a Linux CMake configure for focused validation, but configuration is blocked because the checked-out SDL2 source lacks CMakeLists.txt and XRandR development support is unavailable; no local test executable was built. The PR checks are running on the updated head.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase. Blocking points:

  • New console data in this thread (@ionut27001, swprintf from libSceLibcInternal.sprx on firmware 12.70) contradicts the behaviour this PR implements: %qd, %D, %O, %U and unknown conversions such as %y are printed as literal text (width pads the whole %y/%qd; h/hh are dropped), and %m and %lqd print as text too. The PR formats them the FreeBSD way (-5, 5000000000, y), so the tests and the debt line would pin values the console does not produce. The trailing % case was not cleanly measured. Please rework against that data, or keep main's throw for what is still unmeasured.
  • The head does not compile after the rebase: core/libs/tests/GuestWideFormatting.cpp defines CheckGroupingFlag (lines 149 and 247) and CheckSwprintf (lines 189 and 256) twice, and main calls CheckSwprintf() twice (lines 351, 354).
  • docs/dev/debt/libc.md now has the "narrow printf family" line twice (95, 99) and two "Guest path resolution" lines (104, with a stray .., plus the old one); only the wide printf line should be added.

@MarcZX10

Copy link
Copy Markdown
Contributor Author

Thanks for the probe and review. It confirms behavior in libSceLibcInternal.sprx, which differs from this PR’s target (libc.prx), so it does not establish the target formatter’s behavior. I agree the FreeBSD differential results alone do not justify claiming title compatibility; title-side _Printf evidence is still missing. I corrected the rebase mistakes in head 9c0f6263c6: the duplicate CheckGroupingFlag/CheckSwprintf definitions and call are gone, and the duplicated narrow-printf and guest-path debt entries are removed. The PR description now distinguishes the module evidence and current-head validation. Focused tests still could not be run here; the CMake dependency/configuration blocker is stated above.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the rebase now looks clean (no duplicate helpers or debt lines). The main blocking point is unchanged:

  • FormattingWide.cpp:144-157,216-222 formats %qd, %D/%O/%U and unknown conversions the FreeBSD way, and GuestWideFormatting.cpp:213-244 plus the debt line pin those values. The only console measurement in this thread (@ionut27001, swprintf from libSceLibcInternal.sprx, firmware 12.70) prints all of them as literal text (width pads the whole %y/%qd, h/hh dropped), and %m/%lqd print as text too, while the tests expect an exception for those two. It is a different module than a title's libc.prx, but it is the only hardware data and it contradicts this code; nothing supports the FreeBSD behaviour on the console.
  • To unblock: either implement the measured behaviour (and say in the debt line that it comes from libSceLibcInternal.sprx, not measured on libc.prx), or drop these cases and keep main's throw until _Printf from a title's libc.prx is disassembled or a title reaches them.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 19, 2026
@MarcZX10
MarcZX10 force-pushed the codex/wide-printf-extensions-5293 branch from 9c0f626 to 3c74c99 Compare October 19, 2026 15:42
@MarcZX10 MarcZX10 changed the title fix(libc): support FreeBSD wide printf conversions fix(libc): follow measured wide printf behavior Oct 19, 2026
@MarcZX10

Copy link
Copy Markdown
Contributor Author

I updated the implementation and regression cases to match the reported libSceLibcInternal.sprx payload swprintf output: %q, %D/%O/%U, %m, %lqd, and unknown conversions are preserved as literal text, with width applied to the full text. Incomplete conversions still fail safely because the trailing-% probe read beyond the format terminator. The secure path remains rejecting these forms because it was not measured. The PR title and description now describe this accurately and explicitly state that this does not establish behavior in a title’s bundled libc.prx/Dinkumware _Printf.

At head 3c74c99b673e1513d51fa9aed48511b7118b9a8a, the focused Linux guest_wide_formatting test passes (1/1), conventions pass, and git diff --check passes. The title-side evidence is still absent, as requested in the review; the existing payload measurement is only module-specific evidence.

husseinkad and others added 5 commits October 19, 2026 15:46
…t in the wide printf family

The wide printf family rejected the q length modifier, the D, O and U conversions, unknown conversion characters and a format ending in a percent sign, which FreeBSD 11.4 vfwprintf formats: q is long long, D, O and U are the long forms of d, o and u, an unknown conversion character prints itself with the width and the - flag, and the format ends at a trailing percent sign. swprintf, vswprintf and wprintf now do the same; snwprintf_s keeps rejecting them.
…te pair whole as an unknown conversion

snwprintf_s accepted the q length modifier through the shared parser and now rejects it again. An unknown conversion character that is a surrogate pair is printed as one character, so the width padding no longer splits the pair.
@MarcZX10
MarcZX10 force-pushed the codex/wide-printf-extensions-5293 branch from 3c74c99 to 06bd246 Compare October 19, 2026 15:46
@MarcZX10

Copy link
Copy Markdown
Contributor Author

The review follow-up is now rebased onto current main at d047690c6391577c9add655ae1dc2ed20df177a8; the rebase completed without conflicts and preserved all five commits’ authorship. On the new head 06bd246798, I rebuilt guest_wide_formatting_tests and reran ctest -R guest_wide_formatting --output-on-failure (1/1 passed), tools/check_conventions.py (passed), and git diff --check (passed). The implementation, tests, and debt note reflect the payload libSceLibcInternal.sprx measurements; title libc.prx behavior remains unverified as stated in the PR description.

@github-actions github-actions Bot removed the fix error Closes an issue (Fixes #N) label Oct 19, 2026
@MarcZX10

Copy link
Copy Markdown
Contributor Author

Thanks for approving. I have kept the module-specific measured behavior and the remaining title libc.prx evidence limitation explicit in the PR description and follow-up.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs libc needs rebase Conflicts with main, rebase needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants